Vision system for vehicle

ABSTRACT

A vision system for a vehicle includes an imaging device having an imaging sensor, a camera microcontroller, a display device having a display element, a display microcontroller, and at least one user input selectively actuatable by a user. The imaging device communicates an image signal to the display device via a communication link. The display microcontroller affects the image signal in response to the at least one user input. The camera microcontroller monitors the image signal on the communication link and adjusts a function of the imaging device in response to a detection of the affected image signal. The vision system may adjust a display or sensor of the system in conjunction with a distance detecting system.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application claims the benefit of U.S. provisionalapplication Ser. No. 60/628,709, filed Nov. 17, 2004 (Attorney DocketDON01 P-1188); and Ser. No. 60/614,644, filed Sep. 30, 2004 (AttorneyDocket DON01 P-1177), which are all hereby incorporated herein byreference in their entireties.

FIELD OF THE INVENTION

The present invention relates to imaging systems for vehicles and, moreparticularly, to reverse aid imaging systems with a rearward facingimaging device or camera and a display. However, aspects of the presentinvention are equally suitable for other vehicle imaging systems, suchas side object detection systems, and forward facing imaging systems andthe like.

BACKGROUND OF THE INVENTION

The advent of low cost, reliable imaging devices, based on a variety ofsilicon technologies, and in particular CMOS technology, combined withan improved cost/performance ratio for displays capable of meetingautomotive specifications, and an increasing application rate of videomonitor displays for automotive navigation systems or as part of thedriver interface to a wide variety of vehicle systems, has lead to anincreasing use of cameras or imaging sensors designed to give the drivera view of those areas around the vehicle which are not in the normaldirect field of view of the driver, typically referred to as “blindspots”. These areas include the region close to the front of thevehicle, typically obscured by the forward structure of the vehicle, theregion along the passenger side of the vehicle, the region along thedriver side of the vehicle rearward of the driver, and the areaimmediately rearward of the vehicle which cannot be seen directly orindirectly through the rear view mirror system. The camera or imagingsensor may capture an image of the rearward (or sideward or other blindspot area) field of view, and the image may be displayed to the driverof the vehicle to assist the driver in backing up or reversing orotherwise driving or maneuvering the vehicle. The use of electroniccameras in these applications significantly increases the driver'sknowledge of the space immediately surrounding the vehicle, which may beof importance prior to and during low speed maneuvers, and thuscontributes to the safe completion of such maneuvers. However, in orderto provide user controls or user inputs to allow the driver or user tocontrol various functions of the camera at the rear or side or front ofthe vehicle, additional wiring or connections or communication links maybe needed between a control and user input in the vehicle and the cameraor imaging device at the rear or side or front of the vehicle.

It is known to provide a headlamp control system having an imagingsensor positioned on a vehicle and having a forward field of view. Thesystem may detect headlamps and taillights in the images captured by theimaging sensor. It is also known to provide a lane departure warningsystem that includes an imaging sensor positioned on a vehicle andhaving a forward field of view. The lane departure warning systemdetects lane markers and the like along the road surface in front of thevehicle and determines when the vehicle is drifting out of the lane, andmay provide an alert to the driver of the vehicle when such drifting isdetected. Such systems typically are separate systems with different,independently operable controls and image processors and imagingsensors.

SUMMARY OF THE INVENTION

The present invention provides an imaging and display system or visionsystem for a vehicle that captures images of a scene occurringexteriorly of the vehicle, such as rearward of the vehicle, and displaysthe captured images at a display device in the vehicle. The imaging anddisplay or vision system includes an imaging device or camera that ispositioned at the vehicle with an exterior field of view (such as arearward field of view) for capturing images of the exterior scene. Thedisplay device is positioned within the vehicle and remote from thecamera and is operable to display the captured images in the vehiclewhere they are readily viewable by the driver or occupant of thevehicle. The vision system includes one or more user inputs at thedisplay device and is operable to adjust or control at least onefunction or mode or feature of the camera in response to actuation ofthe user input or inputs. The vision system is operable to communicateimaging signals from the imaging device to the display device and tocommunicate camera control signals from the display device to theimaging device along common connections or wiring or communication linksbetween the imaging device and the display device.

According to an aspect of the present invention, a vision system for avehicle includes an imaging device having an imaging sensor, a cameramicrocontroller, a display device having a display element, a displaymicrocontroller, and at least one user input selectively actuatable by auser. The user input is selectively actuatable by a user. The imagingdevice communicates an image signal to the display device via acommunication link. The display microcontroller affects the image signalin response to the at least one user input. The camera microcontrollermonitors the image signal on the communication link and adjusts afunction of the imaging device in response to a detection of theaffected image signal.

The imaging sensor may have a field of view exteriorly of the vehicle,such as rearwardly of the vehicle, for capturing an image of the sceneoccurring exteriorly of the vehicle. The vision system may includecontrol circuitry at or near or associated with an interior rearviewmirror assembly of the vehicle or a windshield electronics module oraccessory module of the vehicle.

The display microcontroller may affect the image signal by disabling avideo termination at the display device, and the imaging device may stopcommunicating the image signal when the video termination is disabled.The display microcontroller may selectively apply at least two voltagelevels across the communication link in response to at least two userinputs, and the camera microcontroller may adjust a function of theimaging device in response to a respective one of the voltage levels.

The imaging device may include a video encoder that communicates theimage signal to the display device via the communication link. Thecommunication link may comprise a video plus and a video return wire.

According to another aspect of the present invention, a vision systemfor a vehicle includes an imaging device having an imaging sensor, acamera microcontroller, a display device having a display element, adisplay microcontroller, and at least one user input selectivelyactuatable by a user. The imaging device communicates an image signal tothe display device via a communication link, and the displaymicrocontroller communicates a control signal to the imaging device viathe communication link in response to the user input. The cameramicrocontroller receives the control signal and adjusts a function ofthe imaging device in response to the control signal. The image signaland the control signal utilize a common link between the imaging deviceand the display device.

According to another aspect of the present invention, an imaging anddisplay system or vision system may be operable in combination with orin conjunction with an auxiliary or triggering or initial sensing deviceor system or distance sensing/measuring/determining system. Theauxiliary sensing device or system may provide additional sensing areasto cover blind spots that may not be encompassed by the imaging sensoror camera. Optionally, the auxiliary sensing system, such as ultrasonicsensors, radar, lidar, and the like, may detect an object exteriorly ofthe vehicle, and may determine or measure or detect the distance to theobject, whereby the imaging and display system may be adjusted orcontrolled to provide enhanced imaging of the exterior scene or enhancedprocessing of the captured images or enhanced displaying of the imagesof the exterior scene and the detected object, in response to such anobject detection by the auxiliary sensing system. For example, theimaging and display system may adjust a camera setting to provideenhanced imaging of the detected object, or may adjust the processor orcontrol to provide enhanced processing of a portion of the image datathat corresponds to the region of the scene at which the object wasdetected, or may adjust the display to highlight or enhance thedisplayed images of the detected object or of the region of the displaythat corresponds to the region of the exterior scene at which the objectwas detected.

Therefore, the present invention provides an imaging and display systemor vision system that includes an imaging device and a display deviceconnected via video connections or communication links. The imagingdevice includes user inputs and the imaging and display system isoperable to selectively control or adjust the imaging device in responseto the user inputs. The imaging and display system or vision system isoperable to communicate imaging signals from the imaging device to thedisplay device and to communicate camera control signals from thedisplay device to the imaging device along common connections or wiringor communication links between the imaging device and the displaydevice. The imaging and display system or vision system thus may providevarious optional features or functions or modes without having to changeor add wiring or connections between the imaging device and the displaydevice.

The present invention also provides an imaging system or vision systemfor a vehicle that is operable to capture images of a scene occurringexteriorly of the vehicle, such as forward of the vehicle, and thatcaptures and/or processes selective image data differently than otherimage data to extract and analyze the desired data for differentapplications or systems or accessories. The imaging system includes animaging device or camera that is positioned at the vehicle with anexterior field of view (such as a forward field of view) for capturingimages of the exterior scene. The imaging system may be operable toselectively or intermittently capture and/or process some image data ina manner suitable for a lane detection and/or to capture and/or processother image data in a manner suitable for headlamp detection and/or tocapture and/or process other image data in a manner suitable for ambientlight detection and/or other functions, systems or features or the like.

According to another aspect of the present invention, an imaging systemfor a vehicle includes an image sensor and a control. The image sensoris positioned at a vehicle and has an exterior field of view. The imagesensor is operable to capture images of the exterior field of view andto generate image data. The control is operable to adjust the imagesensor to at least two settings, and to process the image data via atleast two processing techniques. The control is operable to synchronizethe image sensor settings and the processing techniques to extractrespective or appropriate information from the captured images for atleast two applications of the imaging system.

The image sensor may have a field of view forwardly of the vehicle forcapturing an image of the scene occurring forwardly of the vehicle.

The control may adjust the image sensor setting to capture image datasuitable for a particular application, and may process those capturedimages via a processing technique suitable for the particularapplication. The control may adjust the image sensor to other settingsand may correspondingly process those captured images via otherprocessing techniques depending on the desired or appropriate orparticular or respective application or function of the imaging system.For example, the control may selectively or intermittently oroccasionally adjust the image sensor setting and processing technique toextract information from the captured image data for a headlamp controland/or a lane departure warning system and/or a rain sensor and/or anavigational system and/or an ambient light sensor and/or a collisionavoidance system and/or a driving separation indicator and/or a back upaid and/or an object detection system and/or the like.

Therefore, the present invention also provides an imaging system orvision system that includes an imaging sensor and a control that areoperable to provide multiple functions or control of multipleaccessories or the like via selective capturing of image data andprocessing of the captured image data. The control is thus operable toselectively capture images and process image data to provide multiplefunctions or applications with a common image sensor and imageprocessor. The imaging system may synchronize the settings of the imagesensor with the processing techniques applied to the image data by theimage processor, in order to perform and/or optimize two or morefunctions of the imaging system. The present invention thus provides amulti-tasking capability to an image sensor and image processor whileproviding enhanced capturing of the images and processing of the imagedata for the desired functions or applications.

These and other objects, advantages, purposes and features of thepresent invention will become apparent upon review of the followingspecification in conjunction with the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a rear perspective view of a vehicle having an imaging anddisplay system thereon in accordance with the present invention;

FIG. 2 is a plan view of the vehicle of FIG. 1;

FIG. 3 is a block diagram of a camera and display device, showing theconnections or wiring typically used to connect the camera to thedisplay device;

FIG. 4 is a block diagram of an imaging and display system in accordancewith the present invention;

FIG. 5 is a plan view of a vehicle incorporating an imaging system inaccordance with the present invention;

FIG. 6 is a side elevation of a portion of a vehicle embodying animaging system in accordance with the present invention;

FIG. 7 is a block diagram of an imaging system having an imager and acontrol;

FIG. 8 is a block diagram of an imaging system in accordance with thepresent invention, showing different exemplary applications of thecontrol; and

FIG. 9 is another block diagram of the imaging system of the presentinvention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring now to the drawings and the illustrative embodiments depictedtherein, an image capture system or vision system or imaging and displaysystem 10 is positioned at an exterior portion 12 a of a vehicle 12,such as at a rearward portion 12 a of the vehicle 12, and is operable tocapture an image of a scene occurring exteriorly of the vehicle, such asrearwardly of the vehicle, and to display the image at a display deviceor display system 14 of the vehicle which is viewable by a driver of thevehicle (FIGS. 1 and 2). Vision system 10 includes display device orsystem 14 and an imaging system or imaging device 16, which includes animager or image capture device or camera 22 that is directed exteriorlyof the vehicle and has an exterior field of view 17 which preferably atleast partially encompasses a “blind spot” area exteriorly of thevehicle. The images or frames captured by imaging system 16 aredisplayed at a display element 28 of display system 14 to assist thedriver in viewing the blind spot areas. The display system 14 includesone or more user inputs 18 (FIG. 4) that are actuatable by a user of thesystem to control or adjust a function of the imaging system 16, whichis remote from the display system 14. The vision system 10 is operableto communicate imaging signals from the imaging device or system to thedisplay device or system and control signals from the display device orsystem to the imaging device or system along common connections orwiring.

Imaging system or imaging device 16 may be positioned at the exteriorportion of the vehicle and directed generally exteriorly of the vehiclefor capturing images of the exterior scene to assist the driver inmaneuvering or driving the vehicle. Vision system 10 and/or imagingsystem 16 may utilize principles of other vehicle vision or imagingsystems, such as a vision or imaging system of the types disclosed inU.S. Pat. Nos. 6,757,109; 6,717,610; 6,396,397; 6,201,642; 5,550,677;5,670,935; 5,877,897; 5,796,094; 6,097,023; and 6,498,620, and U.S.patent application Ser. No. 09/441,341, filed Nov. 16, 1999 by Schofieldet al. for VEHICLE HEADLIGHT CONTROL USING IMAGING SENSOR (AttorneyDocket DON01 P-770); and/or U.S. provisional application Ser. No.60/614,644, filed Sep. 30, 2004 by Camilleri et al. for CAMERA DISPLAYSYSTEM (Attorney Docket DON01 P-1177); and Ser. No. 60/618,686, filedOct. 14, 2004 by Laubinger for VEHICLE IMAGING SYSTEM (Attorney DocketDON01 P-1183), which are hereby incorporated herein by reference. In theillustrated embodiment, the vision system is operable to capture anddisplay images of the rearward area immediately behind the vehicle, soas to assist the driver in backing up or otherwise driving ormaneuvering the vehicle rearwardly. However, the vision system may beoperable to capture and display images of other areas exteriorly of thevehicle to provide images of other blind spot areas around or near thevehicle, without affecting the scope of the present invention. Thevision system thus may be operable to captures images of the sceneimmediately rearward of the vehicle to assist the driver of the vehiclein backing up or maneuvering the vehicle in reverse. The back up assistsystem may be operable in response to the reverse gear of the vehiclebeing selected.

Preferably, the display element or display device may be located at anupper windshield area of the vehicle so as to be readily viewable by thedriver of the vehicle. The display element may be located at or near theinterior rearview mirror assembly of the vehicle or may be associatedwith the interior rearview mirror assembly of the vehicle. Optionally,the display element may be located at or near an accessory module or podor windshield electronics module of the vehicle or may be associatedwith the accessory module or pod or windshield electronics module of thevehicle.

The imaging system may process the captured image data to detect objectsor the like in the exterior scene. Such data processing may utilizeaspects of the image processing techniques described in U.S. Pat. Nos.6,353,392 and 6,313,454; and/or U.S. patent application Ser. No.10/427,051, filed Apr. 30, 2003 by Pawlicki et al. for OBJECT DETECTIONSYSTEM FOR VEHICLE (Attorney Docket DON01 P-1075); and/or U.S.provisional application Ser. No. 60/638,687, filed Dec. 23, 2004 byHiggins-Luthman for OBJECT DETECTION SYSTEM FOR VEHICLE (Attorney DocketDON01 P-1195), which are hereby incorporated herein by reference. Thedisplay device or element may display the image of the scene or maydisplay the detected objects and/or may display graphic overlays or thelike (such as distance measurements, icons, text or the like) to enhancethe display for viewing by the driver or occupant of the vehicle. Thevision system may be selectively operable to provide different functionsor modes or features as selected by the user via one or more userinputs, as discussed below.

The imaging system 16 of vision system 10 thus may be operable in two ormore modes and/or may provide various functions or features that areselectable by a user of the system (such as the driver of the vehicle)via one or more user inputs at or near or associated with the displaysystem. For example, the imaging system may be selectively operable in acolor mode or a black and white mode or a daytime or nighttime mode, ormay be selectively operable to zoom in on a detected object or to panacross a scene or to adjust the gain, exposure and/or white balance inthe captured images, in response to actuation of a respective user inputby a user within the vehicle. Optionally, the imaging system may beselectively operable to process the images and display the images of theexterior scene in an outline form, where only the outlines of detectedobjects are shown at the display element, such as in a black and whiteline format. The image thus may be manipulated to provide an outlineform and thus may be suitable for display on a low cost display element,such as a dot matrix display or the like, such as the type described inU.S. patent application Ser. No. 10/418,486, filed Apr. 18, 2003 byMcMahon et al. for VEHICLE IMAGING SYSTEM (Attorney Docket DON01P-1070), which is hereby incorporated herein by reference. Also, orotherwise, the vision system may provide user inputs that enable a userto selectively actuate an overlay image to overlay images, such asdistance measurements, text, icons, an image representation of the backof the vehicle, or other overlay images or icons, onto or over the imageof the exterior scene, in order to enhance the display of the exteriorscene for the user.

As shown in FIG. 3, and as is typical for various known imaging devicesand vision systems, there are five connections or wires or leads orcommunication links 20 between the display system 14 and the imagingsystem 16. The connections or links include a camera power connection 20a, a ground connection 20 b, a video + connection 20 c, a video returnconnection 20 d and a shield 20 e. In known reverse aid systems,providing a control of the camera, such as to activate a digital zoom orpan mode for the camera (or to otherwise control or adjust a camerasetting or mode), from the display area (and a user input at or near orassociated with the display device) requires additional wiring for thesecondary communication or communications between the user input of thedisplay device and the camera and, thus, is a costly option for theautomotive market and makes retrofitting a reverse aid system on avehicle very difficult and costly. Controlling or adjusting or actuatingthese camera functions or modes or settings from the display area withinthe vehicle increases the overall system level performance and addsadditional functionality to the system.

As shown in FIGS. 3 and 4, vision system 10 includes the communicationlinks 20 (only the video + connection 20 c and the video returnconnection 20 d are separately shown in FIG. 4 for purposes of clarity)between the imaging system 16 and display system 14. As shown in FIG. 4,the imaging system 16 of vision system 10 includes an imaging sensor orimager 22, a camera microcontroller 24 and a video encoder 26. Theimager or imaging array sensor 22 captures image data indicative of theexterior scene and generates a data signal 22 a and outputs the datasignal to the video encoder 26, which encodes the data and communicatesan encoded video signal 26 a to the display system 14 via the video +and video return connections 20 c, 20 d. Under normal operation, thecamera typically outputs an NTSC video signal on the video + and video −or return connections 20 c, 20 d to the display system 14. This istypically done at about a 30 hertz frame rate. The cameramicrocontroller 24 controls the imager 22 and the video encoder 26, andmonitors (as shown via monitoring link 24 a in FIG. 4) the encoded videosignal 26 a from the encoder 26 to the display system 14. When a changein the signal 26 a is detected by the camera microcontroller, the cameramicrocontroller 24 may adjust or control the imager 22 and/or videoencoder 26 to set the imaging device or system to a different mode orfunction, as discussed below.

Display system or display device 14 includes a display element 28 and adisplay microcontroller 34. The display system 14 receives the encodedvideo signal 26 a from the imaging system 16 via the video + and videoreturn connections 20 c, 20 d, and displays the captured image on thedisplay element 28, such as an LCD panel or the like (or other types ofdisplays or display elements as discussed below). As shown in FIG. 4,display system 14 includes a video buffer and termination 30, a videodecoder 32 and the display element 28. The video buffer and termination30 receives the video signal 26 a and passes the signal to the videodecoder 32, which decodes the encoded video signal and communicates animage signal or RGB signal 32 a to the display element 28. The displayelement 28 displays the captured image within the vehicle where it isreadily viewable by a driver or occupant of the vehicle.

Display system 14 also includes display microcontroller 34 that isoperable to receive a user input signal 18 a from one or more userinputs 18 and to adjust or control the display element 28 and/or toenable or disable the video termination 30 in response to the user inputsignal 18 a. When the video termination is disabled, the cameramicrocontroller 24 detects the change in the signal along the video +and video return connections 20 c, 20 d, and may adjust or control theimager 22 and/or video encoder 26 in response to such detection.

During normal operation of the imaging system, the imaging system 16 mayoutput the NTSC video signal (or any other video signal protocol whetheranalog or digital) on the video + and video − or return connections tothe display system 14. The camera microcontroller 24 may continuously orsubstantially continuously monitor the NTSC video output signal 26 a fora normal output that is about one volt peak into a 75 ohm load (providedat the video buffer and termination 30). When an appropriate user input[such as a button or switch or the like at or near or associated withthe display element or a voice command or a consequent of a user action(such as selection of a reverse gear of the vehicle or the like) or avehicle action or condition or characteristic or status (such as inresponse to the vehicle reaching a particular or threshold forward orreverse vehicle speed or the vehicle encountering a braking condition orthe like) or other input or condition or characteristic or status or thelike] is activated by a user action (such as a user input forselectively adjusting a camera function or mode, such as a digital zoomfunction or pan function or and adjustment of a gain or exposure orwhite balance function or a selection of a color mode or a black andwhite mode or a night vision mode or the like), the displaymicrocontroller 34 disables the video termination 30 at the displaysystem 14 via a signal 34 a. The disabling of the video termination 30at the display system 14 causes the video encoder 26 of the imagingsystem 16 to stop outputting the NTSC video signal 26 a. The cameramicrocontroller 24 senses the change in the output signal or voltagealong the connections 20 c, 20 d, and may control or adjust the imagerto a different or new camera setting in response to such a detectedchange in signal output/voltage, or may control or adjust the videoencoder if appropriate in response to such a detected change in thevideo signal along the connections 20 c, 20 d.

Optionally, when multiple functions or modes are desired, the displaymicrocontroller 34 may disable the video termination and apply aparticular voltage across the connections 20 c, 20 d (via signal 34 a)that corresponds to the particular function or adjustment selected bythe user at the user inputs, and the camera microcontroller 24 may beoperable to determine the voltage applied on the connections and toadjust or control the imager and/or encoder accordingly. The cameramicrocontroller 24 may be preset or programmed to recognize theparticular voltage across the connections and to control or adjust oractivate the respective function or feature or mode in response to suchdetection and recognition. The camera adjustments or settings (such asactivation of a digital zoom feature, a panning feature, a gain orexposure or white balance function or a black and white mode or nightvision mode or the like) thus may be predetermined and based on the DCvoltage level applied across the connections 20 c, 20 d by the displaymicrocontroller 34 and sensed across the connections 20 c, 20 d by thecamera microcontroller 24.

After the camera adjustment is made by the camera microcontroller, thedisplay microcontroller may enable the video termination and remove theapplied DC voltage if applicable, so that the encoder 26 will againcommunicate the video signal 26 a to the display system 14. Optionally,the display microcontroller 34 may wait a predetermined amount of timebefore enabling the video termination again and removing the DC voltagelevel. This delay allows the camera microcontroller enough time to sensethat the termination has been removed, that the new programmed settingshave been uploaded to the imager, and that the settings have takenaffect. Once the delay is completed, the video termination is enabled atthe display, and the camera microcontroller and video encoder resume theNTSC signal to the display system to communicate the capture image datato the display system.

This process may be repeated for invoking and removal of special cameramodes or functions. For example, for different user inputs andassociated functions, the display microcontroller may disable the videotermination and may apply a different voltage to the connections 20 c,20 d. The camera microcontroller may then determine the voltage in theconnections 20 c, 20 d and may adjust the imager and/or video encoderappropriately and in response to the detected voltage. The number ofspecial modes of operation or functions that are desired to control oractivate/deactivate at the display will dictate the number of voltagesthat are to be resolved by the camera microcontroller at the camera inorder to activate or control or adjust the multiple camera settings fromthe display inputs. For example, if four different functions or modesand associated user inputs were desired, the display microcontroller mayapply different voltages, such as at ¼ volt increments or the like, forthe different inputs and functions, and the camera microcontroller maybe operable to discern or resolve the different voltages and control oradjust the imager and/or video decoder according to the detectedvoltage.

The vision system of the present invention thus provides user selectablecontrol or adjustment of an exterior facing camera via user inputs at aninterior display of the vehicle, without requiring additional wiring forthe connection between the inputs and the camera. The present inventionthus is highly suitable for retrofitting imaging and display systems orvision systems and/or selecting an appropriate or desired camera and/ordisplay device or system that provides the desired features withoutconcerns as to the wiring or connections between the camera and imagingsystem and the display element and display system. A user may select anoptional camera that has desired features and may install the camera atthe vehicle and connect to the existing wiring, which may be the basewiring for a base camera that may not have the features of the newcamera. The present invention thus provides for upgrading of a cameraand/or display and/or vision system either as an option installed at thevehicle assembly plant, or as an aftermarket device, without the costand difficulties associated with rewiring or wiring new connectors orlinks between a user input or inputs and the camera. Different optionalcameras and display systems and imaging systems thus be selected by anowner of a vehicle and may be readily installed in the vehicle withouthaving to re-wire the vehicle or provide complicated communication linksbetween the camera and the display device or system. The presentinvention thus allows for selection of a camera and display device orsystem having the desired features or content, without the costs anddifficulties of implementing the high content cameras, since anyselected camera and display device or system may utilize the sameconnections or communication links.

Optionally, the vision system may include or may be associated with anultrasonic or radar device that determines the distance from the rear ofthe vehicle to a detected object. Optionally, other means fordetermining the distance to a detected object may be implemented, suchas the means described in U.S. patent application Ser. No. 10/427,051,filed Apr. 30, 2003 by Pawlicki et al. for OBJECT DETECTION SYSTEM FORVEHICLE (Attorney Docket DON01 P-1075); and/or Ser. No. 11/105,757,filed Apr. 14, 2005 by Schofield et al. for IMAGING SYSTEM FOR VEHICLE(Attorney Docket DON01 P-1208); and/or U.S. provisional application Ser.No. 60/607,963, filed Sep. 8, 2004 by Schofield for IMAGING SYSTEM FORVEHICLE (Attorney Docket DON01 P-1170), which are hereby incorporatedherein by reference. The vision system may detect the distance toobjects in various zones or regions behind the vehicle and may provide agraphic or video overlay of the displayed image to indicate to thedriver the distance to one or more detected objects. The overlay may beenhanced, such as by flashing or the like, to enhance the viewability ofthe overlay, such as when a hazardous condition is encountered, such aswhen the vehicle is within a threshold distance to a detected objectrearward of the vehicle and in the vehicle's path of travel. The graphicoverlay may be initiated or activated in response to the vehicle beingshifted into the reverse gear or may be selectively activated via a userinput or the like, without affecting the scope of the present invention.

Optionally, the imaging system and/or camera and/or display system maybe operable in conjunction with or in combination with a distancesensing/measuring/determining system, such as an ultrasonic sensingsystem or the like. For example, and with reference to FIG. 5, animaging and display system or vision system 10′ may include an imagingsensor or camera or imaging system 16′, which may be generally centrallypositioned at the rear portion of the vehicle 12′, and one or moresecondary or auxiliary sensing devices, such as ultrasonic sensors 36,which may be also positioned at the rear portion of the vehicle. In theillustrated embodiment, an ultrasonic sensor 36 may be positioned at ornear each rear corner region 12 b′ of the vehicle 12′. Typically, animaging sensor or camera may provide a wide angle field of view (showngenerally at 38 in FIG. 5) rearward of the vehicle that is approximatelya 120 degree field of view. Such a field of view may not fully encompassthe regions immediately rearward of the vehicle and at or toward theside corners of the vehicle, whereby blind spots may still exist inthose regions. In order to provide enhanced object detection of objectsthat may be positioned at those regions, such as when the vehicle isbacking up, the ultrasonic sensors 36 may be operable to sense or detectobjects in those regions. The imaging and display system may thendisplay an indication at the display device or display system 14′ of adetected object in response to such detection by the ultrasonic sensors.For example, the display may provide an iconistic display of the objectsuperimposed upon the video image of the rearward scene as captured bythe rearward facing camera, or may provide a flashing of the display atthe image on the screen and in the region or regions of the detectedobject, and/or may provide a graphical overlay at the regions of thedisplay that represent the detected object to indicate that an objecthas been detected in one of the rear corner regions or elsewhererearward of the vehicle.

Optionally, an ultrasonic sensing system may include a plurality ofsensors, such as three or more ultrasonic sensors, positioned and spacedacross a rear portion of the vehicle, such as along a rear fender of thevehicle, or along a rear lift gate of the vehicle, or along a rearspoiler of the vehicle, or along a light bar associated with the licenseplate of the vehicle or the like. For example, and as shown in FIG. 5,the sensors may include two side or corner sensors 36 and one or moresensors 40 (such as two sensors 40 in FIG. 5) interspaced across therear portion of the vehicle and between the corner sensors 36.Preferably, a light bar is provided at the license plate of the vehicleand includes the imaging sensor or camera, the ultrasonic sensors, andpreferably a light for illuminating the license plate of the vehicle,such that the imaging sensor and ultrasonic sensors may be provided as amodule at the rear of the vehicle.

The ultrasonic sensors 36, 40 may provide adjacent and overlappingsensing regions to provide a sensing range that encompassessubstantially the entire region behind the vehicle and about eight feetor thereabouts rearward of the vehicle. The imaging sensor or camera 16′may capture images of the rearward exterior scene that is encompassed bythe field of view of the camera, and the display may display images ofthe scene for viewing by the driver of the vehicle when the vehicle isbacking up. The display may be adjusted or modified or enhanced in oneor more of the regions associated with one or more of the ultrasonicsensors in response to a detection of an object by one or more of theultrasonic sensors.

For example, the display may be divided into four zones, each zone ofthe display corresponding to a zone or region covered by a respectiveultrasonic sensor. If that ultrasonic sensor detects an object, then thecorresponding zone of the display may be adjusted or enhanced toindicate to the driver of the vehicle that an object is detected in aparticular zone or region rearward of the vehicle. For example, one ormore of the zones in the display may flash or modulate or may changeintensity to indicate that an object has been detected in the region orzone rearward of the vehicle that corresponds to the enhanced orflashing or modulating display zone. Optionally, the display may providea graphic overlay, such as a measurement bar or the like, to show adistance to a detected object.

Optionally, the ultrasonic sensor or auxiliary sensing device mayprovide a triggering or initial sensing function, and the imaging systemmay adjust a camera setting or characteristic in response to a detectionof an object by one or more of the ultrasonic sensors. For example, thecamera may be controlled or adjusted to electronically zoom to theappropriate zone or region of the captured image in response to adetection of an object by a respective ultrasonic sensor. Because thecaptured image may have distortion due to the wide angle lens or opticof the imaging system and camera, it may be desirable to adjust theimaging sensor or camera to provide enhanced imaging of an objectdetected in the field of view. The camera thus may be zoomed in on theappropriate zone or region to provide enhanced imaging of that zone orregion and to provide reduced distortion of that zone or region and,thus, enhanced imaging and reduced distortion of the detected objectwhen it is displayed on the display. The imaging and display system mayoptionally provide a dynamic overlay at the display to highlight thedetected object in the display, and may display the distance (which maybe detected or determined in response to the ultrasonic sensing devices)to the object.

Optionally, the imaging system may process particular zones or regionsof the image data corresponding to the exterior scene more than otherzones or regions, such as by utilizing aspects of the object detectionsystem described in U.S. patent application Ser. No. 10/427,051, filedApr. 30, 2003 by Pawlicki et al. for OBJECT DETECTION SYSTEM FOR VEHICLE(Attorney Docket DON01 P-1075); and/or Ser. No. 11/105,757, filed Apr.14, 2005 by Schofield et al. for IMAGING SYSTEM FOR VEHICLE (AttorneyDocket DON01 P-1208); and/or U.S. provisional application Ser. No.60/607,963, filed Sep. 8, 2004 by Schofield for IMAGING SYSTEM FORVEHICLE (Attorney Docket DON01 P-1170), which are hereby incorporatedherein by reference. Optionally, the imaging system may processparticular zones or regions in response to a detection of an object byone or more of the ultrasonic sensors. For example, if one of theultrasonic sensors initially detects an object at the left rear cornerof the vehicle, the control may process the image data that correspondsto that area or region, or may enhance the processing of the image datathat corresponds to that area or region, in order to identify orclassify or detect the detected object in that area or region. Theimaging and display system may then highlight or enhance the display atthe detected object, such as by a graphic overlay at the display or anadjustment or modulation of the display in the region of the detectedobject. The ultrasonic sensors thus may function to detect an object andto direct the image processor to process the image data that correspondsto the area in which the object was detected. The image processor thusmay process the image data for the regions that may have an objecttherein, and may provide reduced processing of other regions, and thusmay provide efficient and enhanced processing of the captured images.

Optionally, the display may be actuated in response to a detection of anobject behind the vehicle when the vehicle is backing up or in reverse.For example, when a driver shifts the vehicle into reverse, theultrasonic sensors at the rear portion of the vehicle may be activatedto sense the area rearward of the vehicle. The imaging and displaysystem may monitor the ultrasonic sensors or may receive an input fromthe ultrasonic sensors or system, and may initially be in a non-activeor non-display mode, where the display is deactivated. When an objectrearward of the vehicle is detected by the ultrasonic sensors, thedetection by the ultrasonic sensor or sensors may trigger the imagingand display system to be activated or set to an active mode, whereby theimaging sensor may begin capturing images of the rearward scene and thedisplay may begin displaying images of the rearward scene (which mayinclude enhanced imaging or processing or display or highlighting of thedetected object such as discussed above). The activation of the displaymay provide an alert function to the driver of the vehicle that anobject is detected behind the vehicle and/or in the path of the vehicle,so as to draw the driver's attention to the activated display and to theobject being displayed in the display.

Optionally, the imaging and display system and ultrasonic sensing systemmay be implemented in conjunction with a movable video display, such asa slide out or extendable/retractable display screen that extends andretracts from the interior rearview mirror assembly or from an accessorymodule or overhead console or the like within the vehicle cabin and inthe field of view of the driver of the vehicle. For example, the videodisplay may comprise a slide out video display such as described in PCTApplication No. PCT/US03/40611, filed Dec. 19, 2003 by Donnelly Corp. etal. for ACCESSORY SYSTEM FOR VEHICLE (Attorney Docket DON01FP-1123(PCT)); and/or U.S. provisional application Ser. No. 60/630,061,filed Nov. 22, 2004 by Lynam et al. for MIRROR ASSEMBLY WITH VIDEODISPLAY (Attorney Docket DON01 P-1189); Ser. No. 60/667,048, filed Mar.31, 2005 by Lynam et al. for MIRROR ASSEMBLY WITH VIDEO DISPLAY(Attorney Docket DON01 P-1212), which are hereby incorporated herein byreference. The display screen thus may be automatically extended andactivated in response to a detection of an object rearward of thevehicle by the ultrasonic sensors, whereby the extension of the displayscreen provides an alert to the driver so that the driver's attention isreadily drawn to the now extended and activated display screen. Theimaging and display system and ultrasonic sensing system of the presentinvention thus may provide a display only when there is an objectdetected and may provide an alert function to the driver to view thedisplay when an object is detected. In such an application, the videodisplay may not be extended and be activated in response to the shiftinginto reverse, but would extend and activate after detection of an objectexteriorly or rearward of the vehicle by the auxiliary sensor orsensors.

Although described above as ultrasonic sensors or sensing devices, it isenvisioned that the imaging and display system of the present inventionmay be combined with other types of sensing systems or auxiliary orsecondary or triggering or initial sensing devices or sensingtechniques. For example, the imaging and display system may be operablein conjunction with or in combination with radar devices, sonar devices,laser sensing or lidar devices or laser scanning devices or the like,without affecting the scope of the present invention. The initial ortriggering or auxiliary sensing device or devices may initiate ortrigger enhanced imaging or zooming or other characteristics of thecamera, or may initiate or trigger enhanced processing of the capturedimages, or may initiate or trigger enhanced display features or thelike, in response to a detection of an object rearward of the vehicle byone of the initial or triggering or auxiliary sensing devices. Althoughshown and described as being combined with or operating in conjunctionwith a rearward facing camera and associated display, it is envisionedthat aspects of the auxiliary sensing devices or system of the presentinvention may be equally suitable for use in conjunction with sidewardfacing imaging systems, such as side object detection systems or lanechange assist systems or the like, or forward facing imaging systems,such as lane departure warning systems, adaptive speed control, headlampcontrols, rain sensors or the like.

Optionally, and desirably, the image capture device or imaging devicemay be at least partially contained within an imaging module or cameramodule, which includes imaging sensor or imager and a lens positionedwithin a housing which defines a transparent window (which may comprisean at least substantially transparent glass or polycarbonate or acrylic(or other suitable material) window or panel) at the end of lens (suchas described in PCT Application No. PCT/US2003/036177, filed Nov. 14,2003 by Donnelly Corp. for IMAGING SYSTEM FOR VEHICLE, and publishedJun. 3, 2004 as International Publication No. WO 2004/047421 A2(Attorney Docket DON01 FP-1118(PCT)), which is hereby incorporatedherein by reference). The imaging module may include the circuitry andcontrols or camera microcontroller and video encoder for the imagingsensor, such as on one or more printed circuit boards contained withinthe housing.

Optionally, the camera module may comprise a fully sealed module, whichprotects the imaging device and microcontroller and circuitry fromexposure to the elements at the exterior of the vehicle, such as dust,dirt, mud, water, ice, salt and the like. The lens may provide part ofthe sealing of the module. For example, the camera module may include alens that may be purchased or obtained as a subassembly that is separatefrom the imaging sensor and mounted to the module. The lens may loadedinto the module, such as via insertion of the lens or threading of thelens into an aperture or opening in the module wall or adjacent to atransparent cap or cover of the module wall. The lens assembly (which istypically a five element lens set, but may include more or less opticelements without affecting the scope of the present invention) may bepositioned at the opening or aperture or cover such that the lens ispositioned at the appropriate or precise location relative to theimaging sensor or chip to properly focus the image onto the imagingsensor.

Optionally, the lens may be adjustably positioned, such as via threadinginto a threaded opening, to precisely position the lens to focus theimage onto the imaging sensor. Once positioned at the precise orappropriate location, the lens may be retained or secured in the preciseposition, and the module may then be sealed with a back plate and gasketto seal the lens and imaging sensor and associated microprocessor andcircuitry within the module. The module thus provides enhancedpositioning of the lens relative to the imaging sensor, since the lensmay be readily adjusted to provide proper focusing relative to a fixedimaging sensor.

The module may include a transparent cap or cover through which thefield of view of the imaging sensor and lens is directed. Thetransparent cap may comprise a molded polycarbonate material or the likeand may provide a substantially transparent and durable cover at thelens. Optionally, the transparent cap may include an anti-reflectivecoating or a hydrophobic or hydrophilic coating or the like, such as thecoatings described in PCT Application No. PCT/US2003/036177, filed Nov.14, 2003 by Donnelly Corp. for IMAGING SYSTEM FOR VEHICLE (AttorneyDocket DON01 FP-1118(PCT)), which is hereby incorporated herein byreference. The transparent cap thus may provide functionality andsealing to the camera module.

Because the lens assembly may be selected and positioned at the imagingsensor as a separate sub-assembly or as an aftermarket sub-assembly, thelens assembly may be selected to provide the desired effect depending onthe particular application of the camera module. The lens may then beassembled to the module and the appropriate module, with the desiredlens and features or content, may be mounted to the vehicle andconnected to the existing connector (such as a five wire connector asdescribed above). The display system may further process the image dataor encoded signal to digitally correct for distortion in the image dueto lens distortion and the like.

The present invention thus provides a customized imaging system thatprovides various features or functions or modes or content to theimaging device or system without requiring costly communication or datawires connecting between the imaging device and the user inputs withinthe vehicle. The present invention thus provides an vision system thatis suitable for economically configuring the system to the desiredcontent for a particular application of the imaging system. A user thusmay select a vision system with a particular camera and/or display, withlittle or no affect on the wiring or connections between the camera andthe display.

Optionally, the imaging system may be operable to function as a lanedeparture warning (LDW) system utilizing image processing of the imagescaptured by the rearward facing imaging sensor or camera. The imagingsystem may include controls and/or circuitry for operating as such alane departure warning system and thus may process the images to detectthe lane markers and the like along the road surface, or the imagingsystem may provide image data to a separate image processor ormicrocontroller for processing the image data to detect the lane markersand the like along the road surface. The imaging system may utilizeimage processing techniques such as those described in U.S. patentapplication Ser. No. 10/427,051, filed Apr. 30, 2003 by Pawlicki et al.for OBJECT DETECTION SYSTEM FOR VEHICLE (Attorney Docket DON01 P-1075);and/or Ser. No. 11/105,757, filed Apr. 14, 2005 by Schofield et al. forIMAGING SYSTEM FOR VEHICLE (Attorney Docket DON01 P-1208); and/or U.S.provisional application Ser. No. 60/607,963, filed Sep. 8, 2004 bySchofield for IMAGING SYSTEM FOR VEHICLE (Attorney Docket DON01 P-1170);and/or Ser. No. 60/638,687, filed Dec. 23, 2004 by Higgins-Luthman forOBJECT DETECTION SYSTEM FOR VEHICLE (Attorney Docket DON01 P-1195),which are hereby incorporated herein by reference.

The imager or imaging sensor 22 of the imaging system 16 of the imagingsystem 10 may comprise an imaging array sensor or a pixelated imagingarray, such as a multi-pixel array such as a CMOS sensor or a CCD sensoror the like, such as the types disclosed in commonly assigned U.S. Pat.Nos. 5,550,677; 5,670,935; 5,796,094; 5,877,897; 6,097,023; 6,498,620;and 6,690,268, and U.S. patent application Ser. No. 09/441,341, filedNov. 16, 1999 by Schofield et al. for VEHICLE HEADLIGHT CONTROL USINGIMAGING SENSOR (Attorney Docket DON01 P-770), which are herebyincorporated herein by reference, or such as an extended dynamic rangecamera, such as the types disclosed in PCT Application No.PCT/US2003/036177, filed Nov. 14, 2003 by Donnelly Corp. for IMAGINGSYSTEM FOR VEHICLE (Attorney Docket DON01 FP-1118(PCT)), which is herebyincorporated herein by reference. For example, the imaging sensor maycomprise a CMOS camera, such as the OV7930 single chip CMOS color NTSCcamera available from OmniVision Technologies Inc. of Sunnyvale, Calif.Such color cameras may have the performance characteristics identifiedabove and may additionally provide RGB and/or YCrCb video signals.Preferably, the color video camera operates at a minimum illumination(3000 K) of less than about 5 lux at f1.2, more preferably of less thanabout 3 lux at f1.2, and most preferably less than about of less thanabout 2 lux at f1.2. Such CMOS imaging sensors typically may have a peaksensitivity in the near infrared range, such as at approximately 850 nmto 900 nm.

Such pixelated imaging sensors may include a plurality of pixels, withat least some of the pixels masked or covered with a particular colorfilter, such that the individual pixels function to capture a particularcolor, such as red, green and blue colors or the like, such as disclosedin U.S. Pat. Nos. 5,550,677; 5,670,935; 5,796,094; 6,097,023; and/or6,498,620, referenced above. For example, the imaging sensor maycomprise an individual blue or a green or a red color filter over eachpixel element of the CMOS multi-pixel element array. The imaging sensoris thus operable to provide color images to the display. Such RGBfilters enable the capture of a color image by the CMOS detector, butnecessarily result in a reduced or decreased low light level sensitivityfor a color camera compared to a monochromatic or black and whitecamera. Optionally, and preferably, the imaging sensor may be capable ofselectively operating in either a color mode, in which a color image maybe displayed at display element 28 of display system 14, or amonochromatic or black and white mode, in which a monochromatic or blackand white image may be displayed at display element 28 of display system14, such as by utilizing aspects of the imaging sensor disclosed in U.S.Pat. No. 6,498,620; and/or PCT Application No. PCT/US2003/036177, filedNov. 14, 2003 by Donnelly Corp. for IMAGING SYSTEM FOR VEHICLE (AttorneyDocket DON01 FP-1118(PCT)), which are hereby incorporated herein byreference.

Although described as a CMOS type camera, clearly other types of imagingarrays or imaging sensors or cameras may be implemented with the imagingsystem of the present invention. For example, the imaging sensor maycomprise a CCD or other type of sensor, without affecting the scope ofthe present invention. Preferably, the selected imaging sensor has a lowdark current and thus provides enhanced ruggedness and enhancedperformance at higher temperatures. Optionally, the dark current (thecurrent through the pixels when they are not sensing light) may be usedto detect the temperature at the imaging sensor, such as described inU.S. patent application Ser. No. 11/105,757, filed Apr. 14, 2005 bySchofield et al. for IMAGING SYSTEM FOR VEHICLE (Attorney Docket DON01P-1208); and U.S. provisional application Ser. No. 60/607,963, filedSep. 8, 2004 by Schofield for IMAGING SYSTEM FOR VEHICLE (AttorneyDocket DON01 P-1170), which is hereby incorporated herein by reference.

Optionally, the imaging device or system may communicate the videosignals to the display device or system via other types of signalcommunicating means. For example, the imaging device or system maycommunicate the video signals to the display device or system via anLVDS output of the imaging device or system. Optionally, the imagingsystem and the display system may share common components or circuitryor a common microprocessor to reduce components and cost of the visionsystem.

Referring now to FIG. 6, an image capture system or imaging system 110is positioned at a vehicle, such as at or in an interior portion of thevehicle, such as at or in an accessory module or pod or attachment 112of the vehicle, and is operable to capture an image of a scene occurringexteriorly of the vehicle, such as forwardly of the vehicle. Imagingsystem 110 includes an image capture device or imaging device or imagingsystem or camera or sensor 114 that is directed exteriorly of thevehicle and has an exterior field of view 116. The images or framescaptured by image sensor 114 are processed by a control 118 to detectobjects or items of interest in the captured images. The control mayprocess the captured image data to determine if headlamps or taillightsof other vehicles are present in the images, such as for a headlampcontrol system, and/or the control may process the captured images todetect lane markers along the road surface, such as for a lane departurewarning system, and/or may process the image data to detect othercharacteristics or objects, as discussed below (such as by utilizingaspects of the imaging system described in U.S. provisional applicationSer. No. 60/618,686, filed Oct. 14, 2004 (Attorney Docket DON01 P-1183),which is hereby incorporated herein by reference in its entirety). Thecontrol may synchronize the processing techniques and the camera orimage sensor settings to enhance and/or optimize processing of imagesfor the particular system or function, as also discussed below.

Imaging sensor 112 may be positioned at the vehicle and directed ororiented with a field of view generally exteriorly of the vehicle forcapturing images of the exterior scene for processing by the control, asdiscussed below. Imaging system 110 may utilize principles of othervehicle vision or imaging systems, such as a vision or imaging system orcontrol of the types disclosed in U.S. Pat. Nos. 6,757,109; 6,717,610;6,396,397; 6,201,642; 6,353,392; 6,313,454; 6,396,397; 5,550,677;5,670,935; 5,796,094; 5,877,897; 6,097,023; and 6,498,620, and U.S.patent application Ser. No. 09/441,341, filed Nov. 16, 1999 by Schofieldet al. for VEHICLE HEADLIGHT CONTROL USING IMAGING SENSOR (AttorneyDocket DON01 P-770), and Ser. No. 10/427,051, filed Apr. 30, 2003 byPawlicki et al. for OBJECT DETECTION SYSTEM FOR VEHICLE (Attorney DocketDON01 P-1075), which are all hereby incorporated herein by reference. Ina preferred embodiment, the imaging system 112 may include a lenselement or optic between the imaging device 114 and the exterior scene.The optic may comprise an asymmetric optic, which focuses a generallycentral portion of the scene onto the imaging device, while providingclassical distortion on the periphery of the scene or field of view.

In the illustrated embodiment, the image sensor 112 is mounted at or inan accessory module or windshield electronics module or mirrorattachment or pod 120 and is arranged to have a field of view 116forward of the vehicle, such as through the windshield of the vehicleand preferably through a portion of the windshield that is wiped by awindshield wiper of the vehicle. The image sensor 112 thus may captureimages of a forward scene as the vehicle is traveling forwardly along aroad or street or highway or the like. Optionally, the imaging device orsensor may be positioned elsewhere, such as at or in the interiorrearview mirror assembly 122, or at or in an accessory module orwindshield electronics module or the like (as discussed below), withoutaffecting the scope of the present invention. Optionally, the imagesensor and imaging system may be operable to capture images of otherareas exteriorly of the vehicle to provide image data of other areasaround or near the vehicle, without affecting the scope of the presentinvention. The imaging system thus may be operable to capture images ofthe scene sidewardly of the vehicle for a blind spot or side objectdetection or lane change assist systems or immediately rearward of thevehicle to detect objects rearward of the vehicle, such as for assistingthe driver of the vehicle in backing up or maneuvering the vehicle inreverse.

As shown in FIG. 7, control 118 may control the imager or image sensor114 via a control signal 118 a, while the control may receive image datafrom the image sensor via an image data signal 114 a. The control ormicroprocessor 118 is operable to process the image data generated bythe image sensor 114 to analyze the image data and detect objects orlight sources of interest in the captured image. The control or imagingsystem may selectively process the captured image data to detect objectsor light sources or lane markers or the like in the exterior scene, andmay process particular portions or regions of the captured image inaccordance with the respective function of the control, as discussedbelow.

The control may be operable to control various accessories (or togenerate an output to a control or circuitry of one or more accessories)in response to the image processing. For example, the control maycontrol or adjust a headlamp high-low beam setting in response to adetection of headlamps or taillights in the captured images, and/or mayactuate an alert device, such as a visible display or audible alert orthe like, in response to a detection of the vehicle drifting out of itslane along the road, and/or may adjust an interior lighting or displayintensity of a light or display within the vehicle in response to anambient light detection, and/or may generate other outputs oractuate/control other accessories in response to detections of otherobjects or light sources of interest via such image processing.

The imaging system thus may be operable to function as or in conjunctionwith a lane departure warning system, and may generate an alert to thedriver of the vehicle in response to a detection that the vehicle isdrifting or moving out of a detected lane along the road. Such anapplication may utilize principles of systems of the types described inU.S. Pat. Nos. 6,353,392 and 6,313,454; and/or U.S. patent applicationSer. No. 10/427,051, filed Apr. 30, 2003 by Pawlicki et al. for OBJECTDETECTION SYSTEM FOR VEHICLE (Attorney Docket DON01 P-1075); Ser. No.11/105,757, filed Apr. 14, 2005 by Schofield et al. for IMAGING SYSTEMFOR VEHICLE (Attorney Docket DON01 P-1208); and/or Ser. No. 10/209,173,filed Jul. 31, 2002 by Schofield for AUTOMOTIVE LANE CHANGE AID, nowU.S. Pat. No. 6,882,287 (Attorney Docket DON01 P-1016), and/or U.S.provisional application Ser. No. 60/607,963, filed Sep. 8, 2004 bySchofield for IMAGING SYSTEM FOR VEHICLE (Attorney Docket DON01 P-1170);and/or Ser. No. 60/638,687, filed Dec. 23, 2004 by Higgins-Luthman forOBJECT DETECTION SYSTEM FOR VEHICLE (Attorney Docket DON01 P-1195),which are hereby incorporated herein by reference. Optionally, theimaging system may capture and process images for a headlamp controlapplication, and may utilize principles of systems of the typesdescribed in U.S. Pat. Nos. 6,824,281; 5,796,094; 6,097,023; 5,320,176;and 6,559,435, which are hereby incorporated herein by reference, and/ormay capture and process images for a rain sensing function or rainsensor application, and may utilize principles of the systems of thetypes described in U.S. Pat. Nos. 6,824,281; 6,320,176; 6,353,392;6,313,454; 6,516,664; 6,341,523; and 6,250,148; and/or in U.S. patentapplication Ser. No. 10/348,514, filed Jan. 21, 2003 by Lynam for RAINSENSOR MOUNTING SYSTEM (Attorney Docket DON01 P-1057), which are allhereby incorporated herein by reference, and/or may capture and processimages for an ambient light detection or the like, and may utilizeprinciples of systems of the types described in U.S. Pat. Nos. 5,550,677and/or 5,670,935, which are hereby incorporated herein by reference,and/or may capture and process images for a collision avoidance systemor vehicle separation system, and may utilize principles of systems ofthe types described in U.S. Pat. Nos. 6,411,204; 6,396,397; 6,124,647;6,291,906; and 6,534,884, and/or U.S. patent application Ser. No.10/422,512, filed Apr. 24, 2003 by Schofield for DRIVING SEPARATIONDISTANCE INDICATOR (Attorney Docket DON01 P-1072), which are all herebyincorporated herein by reference, and/or may capture and process imagesfor a navigational system, and may utilize principles of systems of thetypes described in U.S. Pat. Nos. 6,477,464; 5,924,212; 4,862,594;4,937,945; 5,131,154; 5,255,442; 6,678,614; and/or 5,632,092, and/orU.S. patent application Ser. No. 10/456,599, filed Jun. 6, 2003 byWeller et al. for INTERIOR REARVIEW MIRROR SYSTEM WITH COMPASS (AttorneyDocket DON01 P-1076); Ser. No. 10/645,762, filed Aug. 20, 2003 by Tayloret al. for VEHICLE NAVIGATION SYSTEM FOR USE WITH A TELEMATICS SYSTEM(Attorney Docket DON01 P-1103); Ser. No. 11/105,757, filed Apr. 14, 2005by Schofield et al. for IMAGING SYSTEM FOR VEHICLE (Attorney DocketDON01 P-1208); and Ser. No. 10/422,378, filed Apr. 24, 2003, now U.S.Pat. No. 6,946,978 (Attorney Docket DON01 P-1074); and/or PCTApplication No. PCT/US03/40611, filed Dec. 19, 2003 by DonnellyCorporation et al. for ACCESSORY SYSTEM FOR VEHICLE (Attorney DocketDON01 FP-1123(PCT)); and/or PCT Application No. PCT/US04/015424, filedMay 18, 2004 by Donnelly Corporation et al. for MIRROR ASSEMBLY FORVEHICLE (Attorney Docket DON01 FP-1150(PCT)), and/or U.S. provisionalapplication Ser. No. 60/607,963, filed Sep. 8, 2004 by Schofield forIMAGING SYSTEM FOR VEHICLE (Attorney Docket DON01 P-1170), which are allhereby incorporated herein by reference. Other accessories or functionsor applications may also or otherwise be provided or controlled oradjusted by the imaging system, without affecting the scope of thepresent invention.

The control may adjust one or more characteristics or settings of theimage sensor or camera and/or may operate an iris and/or an optical zoomand/or a digital zoom or the like so that the image sensor is adapted orset to enhance image capturing for a particular function of the imagingsystem or control. Optionally, the control may adjust a focus of theimage sensor (such as via adjustment of the lens or optic) to provide aclear captured image focused on the desired or appropriate objects forthe particular application of the imaging system.

The imager or imaging sensor 114 of the imaging system 110 may comprisean imaging array sensor or a pixelated imaging array, such as amulti-pixel array such as a CMOS sensor or a CCD sensor or the like,such as the types disclosed in commonly assigned U.S. Pat. Nos.5,550,677; 5,670,935; 5,796,094; 5,877,897; 6,097,023; 6,498,620; and6,690,268, and U.S. patent application Ser. No. 09/441,341, filed Nov.16, 1999 by Schofield et al. for VEHICLE HEADLIGHT CONTROL USING IMAGINGSENSOR (Attorney Docket DON01 P-770), which are hereby incorporatedherein by reference. For example, the multi-pixel array may comprise amulti-pixel array having a 640×480 pixel array or the like. In order toreduce the computational load on the processor, only a desired subset orzone of the multi-pixel array frame may be chosen to be processed, suchas for a particular application of the imaging system. For differentapplications, the objects or light sources of interest may be differentobjects or categories or classifications of objects and/or may betypically located at different regions of the captured image. Forexample, for a headlamp control application or function, the objects ofinterest are light sources, particularly headlamps and taillights, thatare typically located at a generally central band or region of interestacross the captured images, while for a lane departure warningapplication or function, the objects of interest are lane markers orroad edges or the like that are typically located along a lower regionof interest of the captured images (such as along the lower about 30percent of the captured image or thereabouts). For ambient lightdetection, the light or region of interest would typically be along theupper region of the captured images (such as about the upper 20 percentof the captured image or thereabouts). The desired or appropriate orchosen subset or zone or target area of the multi-pixel array frame maybe processed for the particular application to reduce processing orcomputational load when additional processing of other subsets or zonesmay not enhance the performance of the imaging system for thatparticular application.

Also, for different applications, it may be desirable to capture therespective images with the image sensor being at a different sensitivityor setting. For example, a camera setting or characteristic, such as anexposure time, an amplification, a gain, a white balance and/or thelike, of the image sensor may be adjusted for capturing images tooptimize the captured image and image data for a particular applicationor function. Amongst other things, the sensitivity of the image sensorcan be adjusted by adjusting the integration time of the pixels and theamplification stage in reading out pixels. This allows the image sensorto be adapted to various lighting conditions at the observed exteriorscene. In a closed loop camera control scheme, the image sensorsensitivity can be adjusted as determined by an application module sothat the image brightness meets the application specific requirements.For example, the control may adjust the image sensor to be at anincreased sensitivity for detecting headlamps and taillights duringnighttime conditions, yet may decrease the sensitivity for detectinglane markers along the road surface, particularly during nighttimeconditions when the headlamps are activated and the road is thus wellilluminated. Other adjustments or settings or characteristics, such as adesired or selected zone of interest in a frame or an optical zoom ordigital zoom or panning of or through a frame or the like, may be madeor set or adjusted to optimize the image sensor for capturing one ormore images for a particular function or accessory control.

The control thus may adjust a characteristic or setting of the imagingsensor for particular captured images and may process the image data ofthose particular captured images in a manner or processing techniquesuited for a particular application or function of the control, such asfor headlamp control, lane departure warning, rain sensing, vehicledetection, vehicle lighting and/or display control, and/or the like. Thecontrol may synchronize the camera settings for particular capturedimages with the processing technique applied to the image dataassociated with those captured images, and may switch or alternatebetween different settings and processing techniques to provide thedesired processing and analysis and output for the desired applications.

For example, and with reference to FIGS. 8 and 9, the control mayselectively or occasionally or intermittently process images for firstand second applications, such as a lane departure warning system and aheadlamp control or the like. The control thus may adjust a camerasetting or settings or sensitivity or other characteristic or parameterof the image sensor for capturing a first set of images and may processthe image data from the first set of captured images in a first manner,and may generate a first output in response to the first imageprocessing. The control may also adjust a camera setting or settings orsensitivity or other characteristic or parameter of the image sensor forcapturing a second set of images and may process the image data from thesecond set of captured images in a second manner, and may generate asecond output in response to the first image processing. The first andsecond camera settings/characteristics and/or the first and secondprocessing manners or techniques may be different from one another tooptimize the capturing and processing of the image data for theparticular first or second application or function. The control mayfurther adjust the camera settings and image processing techniques for athird application or more applications as desired, without affecting thescope of the present invention. The first and second (and third and moreif applicable) settings and processing techniques are synchronized suchthat each image captured by the image sensor at one of the settings isprocessed by the control or processor at the appropriate orcorresponding processing technique for the associated application orfunction or accessory control.

Multiple applications thus can be operated and controlled using the sameimager and processing resources of the imaging system. In a case whereonly two applications share the system resources and where frames ofimage data are thus processed alternately by the two application modulesusing the same processor, the control may alternate every other imagefor the two applications. For example (and as shown in FIG. 9), images1, 3, 5, etc. may be captured with the image sensor settings at aparticular setting and may be processed according to a particularprocessing technique, while images 2, 4, 6, etc. may be captured withthe image sensor settings at a different setting and/or may be processedaccording to a different processing technique. Clearly, however, otheriterations and alternate capturing and/or processing of images may beimplemented without affecting the scope of the present invention.Although shown as separate applications in FIG. 9, the same imageprocessor may process the different sets of images for the respectivefirst and second applications of the imaging system.

Optionally, such as for cases where the two applications do not have toprocess images at the same frame rate, the frame rate ratio can beadjusted accordingly. For example, if the imager is operable to provideup to 15 frames per second (fps), and the first application needs thecontrol to process five frames per second while the second or otherapplication only needs the control to process one frame per second, thenthe control may process five frames in a row for the first application,and may then process only one frame for the second application. Thisprocessing scheme may be adjusted for any desired frames per second andmay be extended to allow more than two applications to execute on thesystem, without affecting the scope of the present invention.

As shown in FIG. 8, the control or image processor 118 may selectivelyprocess the image data to detect a desired object or light source ofinterest. For example, the control 118 may process image data to detectheadlamps (as shown at 126 in FIG. 8), and may generate an output to aheadlamp control 128 to adjust or control the headlamps of the vehicle(or the control 118 may adjust or control the headlamps of the vehiclein response to the image processing). Optionally, the control 118 mayalso or otherwise selectively process the image data to detect lanemarkers (as shown at 130 in FIG. 8), and may generate an output to alane departure warning system or to an alert device 132 to alert thedriver of a lane drift or the like as the vehicle travels along theroad. Optionally, the control 118 may also or otherwise selectivelyprocess the image data to determine an ambient light level (as shown at134 in FIG. 8), and may generate an output to a display 136 or to aninterior light control 138 or the like to adjust an intensity of thedisplay or lights in response to such image processing. Other capturedimages and/or other objects or light sources or light levels of interestmay also or otherwise be analyzed to provide other outputs to or controlof other functions or accessories of the vehicle, without affecting thescope of the present invention. Each of the image processing techniquesapplied to the image data may be synchronized with the settings orsensitivity of the image sensor to optimize the performance anddetection capabilities of the imaging system for the particularapplications associated with or conducted by the imaging system.

The present invention thus provides an imaging system that mayselectively capture images via an image sensor and may selectivelyprocess the captured images to determine or detect a desired orappropriate characteristic of the captured images. The imaging systemmay synchronize a setting or sensitivity or focus of the image sensorwith an image processing technique so that the captured image data areselectively captured and/or processed in a particular manner for aparticular application of the imaging system. The imaging system thusmay provide multiple functions or applications by utilizing a commonimaging sensor and common image processor, and thus may provide amulti-tasking imaging system at a reduced cost and with reducedcomponents. The imaging system may capture multiple frames sequentially,and may elect (such as in a predetermined manner) the processing ortreatment of the respective frames or sets of frames. The imaging systemthus may focus or harness the processing power to a restricted set ofcriteria for one subset of frames, and to another restricted set ofcriteria for another subset of frames. The imaging system of the presentinvention thus may selectively focus or harness the image sensorcapturing capabilities and the image processing power to enhance theimage capturing and processing of different frames or sets of frames fordifferent applications.

Although shown and described as a forward facing imaging sensor, it isenvisioned that aspects of the present invention may be suitable forselectively capturing and/or processing image data with a rearwardfacing imaging sensor and control or image processor. For example, therearward facing imaging system may be operable to function as a lanedeparture warning (LDW) system utilizing image processing of the imagescaptured by a rearward facing imaging sensor or camera. The imagingsystem may include controls and/or circuitry for operating as such alane departure warning system and thus may process the images to detectthe lane markers and the like along the road surface, or the imagingsystem may provide image data to a separate image processor ormicrocontroller for processing the image data to detect the lane markersand the like along the road surface. The imaging system may utilizeimage processing techniques such as those described in U.S. patentapplication Ser. No. 10/427,051, filed Apr. 30, 2003 by Pawlicki et al.for OBJECT DETECTION SYSTEM FOR VEHICLE (Attorney Docket DON01 P-1075);and/or Ser. No. 11/105,757, filed Apr. 14, 2005 by Schofield et al. forIMAGING SYSTEM FOR VEHICLE (Attorney Docket DON01 P-1208); and/or U.S.provisional application Ser. No. 60/607,963, filed Sep. 8, 2004 bySchofield for IMAGING SYSTEM FOR VEHICLE (Attorney Docket DON01 P-1170);and/or Ser. No. 60/638,687, filed Dec. 23, 2004 by Higgins-Luthman forOBJECT DETECTION SYSTEM FOR VEHICLE (Attorney Docket DON01 P-1195),which are hereby incorporated herein by reference. Optionally, theimaging system may selectively capture and process image data of therearward facing image sensor to provide a back up aid function, such asby utilizing principles described in U.S. Pat. Nos. 5,550,677;5,760,962; 5,670,935; 6,201,642; 6,717,610 and/or 6,757,109, and/or inU.S. patent application Ser. No. 10/418,486, filed Apr. 18, 2003 byMcMahon et al. for VEHICLE IMAGING SYSTEM (Attorney Docket DON01P-1070), and/or in PCT Application No. PCT/US2003/036177, filed Nov. 14,2003 by Donnelly Corporation et al. for IMAGING SYSTEM FOR VEHICLE(Attorney Docket DON01 FP-1118(PCT)), which are all hereby incorporatedherein by reference.

The imager or imaging sensor 114 of the imaging system 110 may comprisean imaging array sensor or a pixelated imaging array, such as amulti-pixel array such as a CMOS sensor or a CCD sensor or the like,such as the types disclosed in commonly assigned U.S. Pat. Nos.5,550,677; 5,670,935; 5,796,094; 5,877,897; 6,097,023; 6,498,620; and6,690,268, and U.S. patent application Ser. No. 09/441,341, filed Nov.16, 1999 by Schofield et al. for VEHICLE HEADLIGHT CONTROL USING IMAGINGSENSOR (Attorney Docket DON01 P-770), which are hereby incorporatedherein by reference, or such as an extended dynamic range camera, suchas the types disclosed in PCT Application No. PCT/US2003/036177, filedNov. 14, 2003 by Donnelly Corp. for IMAGING SYSTEM FOR VEHICLE (AttorneyDocket DON01 FP-1118(PCT)), which is hereby incorporated herein byreference. For example, the imaging sensor may comprise a CMOS camera,such as the OV7930 single chip CMOS color NTSC camera available fromOmniVision Technologies Inc. of Sunnyvale, Calif. Such color cameras mayhave the performance characteristics identified above and mayadditionally provide RGB and/or YCrCb video signals. Preferably, thecolor video camera operates at a minimum illumination (3000 K) of lessthan about 5 lux at f1.2, more preferably of less than about 3 lux atf1.2, and most preferably less than about of less than about 2 lux atf1.2. Such CMOS imaging sensors typically may have a peak sensitivity inthe near infrared range, such as at approximately 850 nm to 900 nm orthereabouts.

Such pixelated imaging sensors may include a plurality of pixels, withat least some of the pixels masked or covered with a particular colorfilter, such that the individual pixels function to capture a particularcolor, such as red, green and blue colors or the like, such as disclosedin U.S. Pat. Nos. 5,550,677; 5,670,935; 5,796,094; 6,097,023; and6,498,620, referenced above. For example, the imaging sensor maycomprise an individual blue or a green or a red color filter over eachpixel element of the CMOS multi-pixel element array. The imaging sensoris thus operable to detect colors, which is useful in determiningwhether a detected object or item is a headlamp or a taillight, such asdescribed in U.S. Pat. No. 5,796,094, referenced above.

Although described as a CMOS type camera, clearly other types of imagingarrays or imaging sensors or cameras may be implemented with the imagingsystem of the present invention. For example, the imaging sensor maycomprise a CCD or other type of sensor, without affecting the scope ofthe present invention. Preferably, the selected imaging sensor has a lowdark current and thus provides enhanced ruggedness and enhancedperformance at higher temperatures. Optionally, the dark current (thecurrent through the pixels when they are not sensing light) may be usedto detect the temperature at the imaging sensor, such as described inU.S. patent application Ser. No. 11/105,757, filed Apr. 14, 2005 bySchofield et al. for IMAGING SYSTEM FOR VEHICLE (Attorney Docket DON01P-1208); and/or U.S. provisional application Ser. No. 60/607,963, filedSep. 8, 2004 by Schofield for IMAGING SYSTEM FOR VEHICLE (AttorneyDocket DON01 P-1170), which is hereby incorporated herein by reference.

Optionally, a method or system may be implemented to transform a CMOScolor imager from its individual color space into a standardized colorprotocol, such as the CIE color space, in order to enhance theidentification of a detected object or light source. The CentreInternationale dEclairage (CIE) is an international organization thatestablishes methods for measuring color and that recommends standardsand procedures for light and lighting, including colorimetry. The colorstandards for colormetric measurements are internationally acceptedspecifications that define color values mathematically. CIE definescolor as a combination of three axes: x, y, and z. CIE color models areconsidered device independent because the colors should not differ,theoretically, from one output device to another if properly calibrated.

The CIE color spaces thus provide the foundation upon whichdevice-independent color and color management are built. Deviceprofiles, device calibration, and gamut mapping may also be needed forcolor management. A device profile describes a device's colorcapabilities including color gamut, color production method, and deviceoperation modes. Device profiles are typically created by color imagingscientists using spectrophotometers, which are instruments that measurethe relative intensities of light in different parts of the visiblespectrum. The measurements are then used to produce the device profile,such as via various algorithms and the like. Device profiles are used bythe color management software to translate color data from one device toanother based upon an independent color space or system for orderingcolors that respects the relationships or similarity among them.

The CMOS color imager of the imaging system of the present invention isused in an automotive environment where all the road scene objects ofinterest that may be typically encountered may be classified within theCIE Color System. Digital imaging systems represent color in any givennumber of ways. The transformation provides an easy method for theidentification of CIE color. Classification of the various objectswithin the road scene can be enhanced using CIE color as well as otherimage processing methods to provide high confidence that any particulardetected object or light source is correctly identified.

The transformation requires an initial characterization of the pixeloutput of the CMOS imager using four primary calibration targets and aspectroradiometer. The CMOS imager is used to measure the “color” of thecalibrated targets. The calibrated targets are also measured using thespectroradiometer. The transformation yields a three by three matrixthat is applied to the x, y, z values of the camera's output to provideCIE color. The CIE color of the detected object or light source may thenbe compared to the classifications or index of the various objects orlight sources of interest to identify the detected object or lightsource.

Therefore, the present invention provides an imaging system thatincludes an imaging device and a control that is operable to selectivelycapture images and process image data to provide multiple functions orapplications of a common image sensor and image processor. The imagingsystem may synchronize the settings of the image sensor with theprocessing techniques applied to the captured image data to optimize twoor more functions of the imaging system. The present invention thusprovides a multi-tasking capability to a common image sensor and imageprocessor while providing enhanced capturing of the images andprocessing of the image data for the desired functions or applications.

Optionally, an imaging sensor or system of the present invention may beassociated with an imaging and display system for displaying thecaptured images to the driver of the vehicle. In such displayapplications, a variety of display means or display devices may beutilized to visually convey information to the driver of the vehicle,without affecting the scope of the present invention. For example, andsuch as described in U.S. Pat. No. 6,477,464 (Attorney Docket DON01P-887), which is hereby incorporated herein by reference, a text displaymay be provided and/or an iconistic display may be provided, such as adisplay readable through the interior rearview mirror reflective elementitself. In this regard, use of a transflective or display on demand(DOD) type display (such as disclosed in commonly assigned, U.S. patentapplication Ser. No. 10/054,633, filed Jan. 22, 2002 by Lynam et al. forVEHICULAR LIGHTING SYSTEM (Attorney Docket DON01 P-962), and in U.S.Pat. Nos. 6,690,268; 5,668,663 and 5,724,187, the entire disclosures ofwhich are hereby incorporated by reference herein), may be preferred.For example, a video display element or a video display screen or aninformation display element can be used (such as an elongatedalphanumeric/multi-pixel/multi-icon display element and/or such as anLCD display or an emitting display element, such as a multi-pixelelectroluminescent display or field emission display or light emittingdiode display (organic or inorganic) or the like) which is disposedwithin the mirror housing of the interior mirror assembly of thevehicle, and located behind the mirror reflective element in the mirrorhousing, and configured so that the information displayed by the displayelement (that is positioned to the rear of the reflector of the mirrorreflective element) is viewable by the driver through the mirrorreflective element. Such a display can be accomplished by partially orwholly removing the reflector in the area of the display or, morepreferably, by providing a display on demand type display, whereby thereflective element comprises a transflective element, as discussedbelow.

Note that other display locations are possible for display of the videoimage or information display, such as a text message or the like, to thedriver or occupant of the vehicle. For example, a video image may bedisplayed on an LCD video screen of flip-down display (such as isdisclosed in U.S. Pat. No. 6,690,268, incorporated above), or on a videoscreen incorporated into the rearview mirror assembly (such as the typesdescribed in U.S. patent application Ser. No. 10/964,512, filed Oct. 13,2004 by Schofield et al. for VEHICLE COMMUNICATION SYSTEM (AttorneyDocket DON01 P-1174), and/or in PCT Application No. PCT/US03/40611,filed Dec. 19, 2003 by Donnelly Corp. et al. for ACCESSORY SYSTEM FORVEHICLE (Attorney Docket DON01 FP-1123(PCT)), which is herebyincorporated herein by reference). Optionally, for example, a videodisplay located in the front instrument panel can be used, or a videodisplay located in an overhead console (such as an overhead accessorymodule or system as described in PCT Application No. PCT/US03/40611,filed Dec. 19, 2003 by Donnelly Corp. et al. for ACCESSORY SYSTEM FORVEHICLE (Attorney Docket DON01 FP-1123(PCT)), which is herebyincorporated herein by reference) can be used, without affecting thescope of the present invention. Alternately, a low cost, multi-pixeldisplay (such as the type disclosed in U.S. patent application Ser. No.10/418,486, filed Apr. 18, 2003 by McMahon et al. for VEHICLE IMAGINGSYSTEM (Attorney Docket DON01 P-1070), which is hereby incorporatedherein by reference), such as a low cost multi-pixel vacuum fluorescentdisplay, a low cost multi-pixel organic light emitting diode (OLED), alow cost multi-pixel field emission display, or any other or similarmulti-pixel light emitting display or the like may be utilized, withoutaffecting the scope of the present invention.

Optionally, the imaging sensor and control, and optionally a displaydevice and/or an audible alert device or speaker or the like, mayinclude a display element positioned at a rear portion of the vehicle,such as at the rear window of the vehicle or the like, on which thecaptured image is projected, such as via a projector within the vehicle.The displayed or projected image may be viewed at the rear portion ofthe vehicle by the driver of the vehicle, such as by viewing the imagein the rearview mirror or by turning to view the rearward image. Theimage thus may be projected as a reverse image or mirror image displaydepending on the particular application or desired viewing of thedisplayed/projected image.

Optionally, the display device and/or an audible alert device or speakermay be positioned at or in or near the interior rearview mirror assemblyof the vehicle. The mirror assembly may comprise a prismatic mirrorassembly, such as a prismatic mirror assembly utilizing aspectsdescribed in U.S. Pat. Nos. 6,318,870; 5,327,288; 4,948,242; 4,826,289;4,436,371; and 4,435,042; and PCT Application No. PCT/US04/015424, filedMay 18, 2004 by Donnelly Corporation et al. for MIRROR ASSEMBLY FORVEHICLE (Attorney Docket DON01 FP-1150(PCT)); and U.S. patentapplication Ser. No. 10/933,842, filed Sep. 3, 2004 (Attorney DocketDON01 P-1166), which are hereby incorporated herein by reference.Optionally, the prismatic reflective element may comprise a conventionalprismatic reflective element or prism or may comprise a prismaticreflective element of the types described in PCT Application No.PCT/US03/29776, filed Sep. 19, 2003 by Donnelly Corp. et al. for MIRRORREFLECTIVE ELEMENT ASSEMBLY (Attorney Docket DON01 FP-1109(PCT)); U.S.patent application Ser. No. 10/709,434, filed May 5, 2004 by Lynam forMIRROR REFLECTIVE ELEMENT (Attorney Docket DON01 P-1152); and Ser. No.10/993,302, filed Nov. 19, 2004 (Attorney Docket DON01 P-1186); and U.S.provisional application Ser. No. 60/525,952, filed Nov. 26, 2003 byLynam for MIRROR REFLECTIVE ELEMENT FOR A VEHICLE (Attorney Docket DON01P-1130), which are all hereby incorporated herein by reference, withoutaffecting the scope of the present invention. A variety of mirroraccessories and constructions are known in the art, such as thosedisclosed in U.S. Pat. Nos. 5,555,136; 5,582,383; 5,680,263; 6,227,675;6,229,319; and 6,315,421 (the entire disclosures of which are herebyincorporated by reference herein), that can benefit from or beimplemented with the present invention.

Optionally, the interior rearview mirror assembly may comprise anelectro-optic or electrochromic reflective element or cell, such as anelectrochromic mirror assembly and electrochromic reflective elementutilizing principles disclosed in commonly assigned U.S. Pat. Nos.6,690,268; 5,140,455; 5,151,816; 6,178,034; 6,154,306; 6,002,544;5,567,360; 5,525,264; 5,610,756; 5,406,414; 5,253,109; 5,076,673;5,073,012; 5,117,346; 5,724,187; 5,668,663; 5,910,854; 5,142,407; and/or4,712,879, which are hereby incorporated herein by reference, and/or asdisclosed in the following publications: N. R. Lynam, “ElectrochromicAutomotive Day/Night Mirrors”, SAE Technical Paper Series 870636 (1987);N. R. Lynam, “Smart Windows for Automobiles”, SAE Technical Paper Series900419 (1990); N. R. Lynam and A. Agrawal, “Automotive Applications ofChromogenic Materials”, Large Area Chromogenics: Materials and Devicesfor Transmittance Control, C. M. Lampert and C. G. Granquist, EDS.,Optical Engineering Press, Wash. (1990), which are hereby incorporatedby reference herein. The mirror assembly may include one or moredisplays, such as the types disclosed in U.S. Pat. Nos. 5,530,240 and/or6,329,925, which are hereby incorporated herein by reference, and/ordisplay-on-demand or transflective type displays, such as the typesdisclosed in U.S. Pat. Nos. 6,690,268; 5,668,663 and/or 5,724,187,and/or in U.S. patent application Ser. No. 11/021,065, filed Dec. 23,2004 by McCabe et al. for ELECTRO-OPTIC MIRROR CELL (Attorney DocketDON01 P-1193); Ser. No. 10/528,269, filed Mar. 17, 2005 (Attorney DocketDON01 P-1109); and/or Ser. No. 10/993,302, filed Nov. 19, 2004 by Lynamfor MIRROR REFLECTIVE ELEMENT FOR A VEHICLE (Attorney Docket DON01P-1186); Ser. No. 10/533,762, filed May 4, 2005 (Attorney Docket DON01P-116); Ser. No. 11/226,628, filed Sep. 14, 2005 by Karner et al. forMOUNTING ASSEMBLY FOR INTERIOR MIRROR (Attorney Docket DON01 P-1236);and/or Ser. No. 10/054,633, filed Jan. 22, 2002 by Lynam et al. forVEHICULAR LIGHTING SYSTEM (Attorney Docket DON01 P-962); PCT ApplicationNo. PCT/US03/29776, filed Sep. 9, 2003 by Donnelly Corp. et al. forMIRROR REFLECTIVE ELEMENT ASSEMBLY (Attorney Docket DON01 FP-1109(PCT));and/or PCT Application No. PCT/US03/40611, filed Dec. 19, 2003 byDonnelly Corp. et al. for ACCESSORY SYSTEM FOR VEHICLE (Attorney DocketDON01 FP-1123(PCT)), and/or U.S. provisional applications, Ser. No.60/681,250, filed May 16, 2005 (Attorney Docket DON01 P-1221); Ser. No.60/690,400, filed Jun. 14, 2005 (Attorney Docket DON01 P-1225); Ser. No.60/695,149, filed Jun. 29, 2005 (Attorney Docket DON01 P-1227), whichare all hereby incorporated herein by reference.

Optionally, the image sensor and control, and optionally a displaydevice and/or alert device, may be positioned at or in an accessorymodule or windshield electronic module of the vehicle, without affectingthe scope of the present invention. The accessory module may compriseany type of accessory module or windshield electronics module orconsole, such as the types described in U.S. patent application Ser. No.10/355,454, filed Jan. 31, 2003 by Schofield et al. for VEHICLEACCESSORY MODULE, now U.S. Pat. No. 6,824,281 (Attorney Docket DON01P-1050); and Ser. No. 10/456,599, filed Jun. 6, 2003 by Weller et al.for INTERIOR REARVIEW MIRROR SYSTEM WITH COMPASS (Attorney Docket DON01P-1076), and/or U.S. Pat. Nos. 6,690,268; 6,250,148; 6,341,523;6,593,565; and 6,326,613, and/or PCT Application No. PCT/US03/40611,filed Dec. 19, 2003 by Donnelly Corp. et al. for ACCESSORY SYSTEM FORVEHICLE (Attorney Docket DON01 FP-1123(PCT)), which are all herebyincorporated herein by reference.

Optionally, the imaging system may be selectively operable andswitchable to provide the desired function and/or display to the driverof the vehicle. Such switching may occur in response to one or more userinputs in the vehicle, such as at the interior rearview mirror system ofthe vehicle or the like. The user inputs may be positioned at or near adisplay, such as along a perimeter region of a flip out or slide outvideo display of a mirror assembly or along a chin or bezel region ofthe mirror assembly, and may comprise buttons, switches, proximitysensors, touch sensors, or the like, without affecting the scope of thepresent invention. For example, the user inputs may comprise buttons orswitches, such as user inputs of the types described in PCT ApplicationNo. PCT/US2003/036177, filed Nov. 14, 2003 by Donnelly Corp. for IMAGINGSYSTEM FOR VEHICLE (Attorney Docket DON01 FP-1118(PCT)) and/or PCTApplication No. PCT/US04/015424, filed May 18, 2004 by DonnellyCorporation et al. for MIRROR ASSEMBLY FOR VEHICLE (Attorney DocketDON01 FP-1150(PCT)), which are hereby incorporated herein by reference,touch sensors or proximity sensing inputs or the like, such as sensorsof the types described in U.S. Pat. Nos. 6,001,486; 6,310,611;6,320,282; 6,627,918; and 5,594,222; and/or U.S. Pat. Publication No.2002/0044065, published Apr. 18, 2002 by Quist et al. for INTERACTIVEAUTOMOTIVE REARVISION SYSTEM (Attorney Docket DON01 P-889); and/or U.S.patent application Ser. No. 10/933,842, filed Sep. 3, 2004 (AttorneyDocket DON01 P-1166); and/or PCT Application No. PCT/US03/40611, filedDec. 19, 2003 by Donnelly Corp. et al. for ACCESSORY SYSTEM FOR VEHICLE(Attorney Docket DON01 FP-1123(PCT)), which are hereby incorporatedherein by reference, or may comprise inputs molded within the bezel ofthe mirror assembly, such as described in U.S. patent application Ser.No. 11/029,695, filed Jan. 5, 2005 (Attorney Docket DON01 P-1192),and/or U.S. provisional application Ser. No. 60/535,559, filed Jan. 9,2004 by Lindahl for MIRROR ASSEMBLY (Attorney Docket DON01 P-134); Ser.No. 60/553,517, filed Mar. 16, 2004 by Lindahl et al. for MIRRORASSEMBLY (Attorney Docket DON01 P-1145); Ser. No. 60/556,259, filed Mar.25, 2004 (Attorney Docket DON01 P-1147), which are hereby incorporatedherein by reference, and/or other types of buttons or inputs at a bezelregion, such as the types described in U.S. provisional applications,Ser. No. 60/690,401, filed Jun. 14, 2005 by Uken for MIRROR ASSEMBLY FORVEHICLE (Attorney Docket DON01 P-1226), and/or Ser. No. ______, filedSep. 22, 2005 by Uken for MIRROR ASSEMBLY WITH USER INPUTS (AttorneyDocket DON01 P-1241), which are hereby incorporated herein by reference,or may comprise membrane type switches, such as described in U.S. patentapplication Ser. No. 11/140,396, filed May 27, 2005 (Attorney DocketDON01 P-1215), and/or U.S. provisional application Ser. No. 60/575,904,filed Jun. 1, 2004 by Uken for MIRROR ASSEMBLY FOR VEHICLE (AttorneyDocket DON01 P-1157); and Ser. No. 60/624,320, filed Nov. 2, 2004(Attorney Docket DON01 P-1185), which are hereby incorporated herein byreference; and/or the like, without affecting the scope of the presentinvention. The switching or inputs may control or adjust the controland/or imaging sensor, such as by utilizing the principles describedabove and in U.S. provisional application Ser. No. 60/614,644, filedSep. 30, 2004 by Camilleri et al. for CAMERA DISPLAY SYSTEM (AttorneyDocket DON01 P-1177), which is hereby incorporated herein by reference.

Therefore, the present invention provides an imaging and display systemthat includes an imaging device and a display device connected via videoconnections or communication links. The imaging device includes userinputs and the imaging and display system is operable to selectivelycontrol or adjust the imaging device in response to the user inputs. Theimaging and display system is operable to communicate imaging signalsfrom the imaging device to the display device and to communicate cameracontrol signals from the display device to the imaging device alongcommon connections or wiring or communication links between the imagingdevice and the display device. The imaging and display system thus mayprovide various optional features or functions or modes without havingto change or add wiring or connections between the imaging device andthe display device.

Changes and modifications to the specifically described embodiments maybe carried out without departing from the principles of the presentinvention, which is intended to be limited only by the scope of theappended claims as interpreted according to the principles of patentlaw.

1. A vision system for a vehicle, said vision system comprising: animaging device having an imaging sensor; a camera microcontroller; adisplay device having a display element; a display microcontroller; atleast one user input selectively actuatable by a user; and wherein saidimaging device communicates an image signal to said display device via acommunication link, said display microcontroller affecting said imagesignal in response to said at least one user input, said cameramicrocontroller monitoring said image signal on said communication linkand adjusting a function of said imaging device in response to adetection of said affected image signal.
 2. The vision system of claim1, wherein said display microcontroller affects said image signal bydisabling a video termination at said display device.
 3. The visionsystem of claim 2, wherein said imaging device stops communicating saidimage signal when said video termination is disabled.
 4. The visionsystem of claim 3, wherein said display microcontroller selectivelyapplies at least two voltage levels across said communication link inresponse to at least two user inputs, said camera microcontrolleradjusting a function of said imaging device in response to a respectiveone of said at least two voltage levels.
 5. The vision system of claim1, wherein said imaging device comprises a video encoder, said videoencoder communicating said image signal to said display device via saidcommunication link.
 6. The vision system of claim 1, wherein saidcommunication link comprises a video plus and a video return wire. 7.The vision system of claim 1, wherein said imaging sensor has a field ofview exteriorly of the vehicle.
 8. The vision system of claim 1, whereinsaid imaging sensor has a field of view rearwardly of the vehicle. 9.The vision system of claim 1, wherein said vision system is operable toadjust an image displayed on said display device in response to a signalfrom a distance sensing device operable to determine a distance to anobject exteriorly of the vehicle.
 10. The vision system of claim 9,wherein said distance sensing device comprises at least one of anultrasonic sensor, a radar device, a sonar device, a laser sensingdevice, a lidar device and a laser scanning device.
 11. The visionsystem of claim 1, wherein said camera microcontroller and said displaymicrocontroller share a common microprocessor.
 12. A vision system for avehicle, said vision system comprising: an imaging device having animaging sensor; a camera microcontroller; a display device having adisplay element; a display microcontroller; at least one user inputselectively actuatable by a user; and wherein said imaging devicecommunicates an image signal to said display device via a communicationlink, said display microcontroller communicating a control signal tosaid imaging device via said communication link in response to said atleast one user input, said camera microcontroller receiving said controlsignal and adjusting a function of said imaging device in response tosaid control signal, said image signal and said control signal utilizinga common link between said imaging device and said display device. 13.The vision system of claim 12, wherein said camera microcontroller andsaid display microcontroller share a common microprocessor.
 14. Thevision system of claim 12, wherein said display microcontrollercommunicates said control signal by disabling a video termination atsaid display device.
 15. The vision system of claim 14, wherein saidimaging device stops communicating said image signal when said videotermination is disabled.
 16. The vision system of claim 15, wherein saiddisplay microcontroller selectively applies at least two voltage levelsacross said communication link in response to at least two user inputs,said camera microcontroller adjusting a function of said imaging devicein response to a respective one of said at least two voltage levels. 17.The vision system of claim 12, wherein said imaging device comprises avideo encoder, said video encoder communicating said image signal tosaid display device via said communication link.
 18. The vision systemof claim 12, wherein said communication link comprises a video plus anda video return wire.
 19. The vision system of claim 12, wherein saidimaging sensor has a field of view exteriorly of the vehicle.
 20. Thevision system of claim 12, wherein said imaging sensor has a field ofview rearwardly of the vehicle.
 21. The vision system of claim 12,wherein said vision system is operable to adjust an image displayed onsaid display device in response to a signal from a distance sensingdevice operable to determine a distance to an object exteriorly of thevehicle.
 22. The vision system of claim 21, wherein said distancesensing device comprises at least one of an ultrasonic sensor, a radardevice, a sonar device, a laser sensing device, a lidar device and alaser scanning device.
 23. A vision system for a vehicle, said visionsystem comprising: an imaging device having an imaging sensor; a cameramicrocontroller; a display device having a display element, said displayelement being at an upper windshield area of the vehicle and viewable bya driver of the vehicle; a display microcontroller; at least one userinput selectively actuatable by a user; and wherein said imaging devicecommunicates an image signal to said display device via a communicationlink, said display microcontroller affecting said image signal inresponse to said at least one user input, said camera microcontrollermonitoring said image signal on said communication link and adjusting afunction of said imaging device in response to a detection of saidaffected image signal.
 24. The vision system of claim 23, wherein saiddisplay microcontroller affects said image signal by disabling a videotermination at said display device, said imaging device stoppingcommunicating said image signal when said video termination is disabled.25. The vision system of claim 24, wherein said display microcontrollerselectively applies at least two voltage levels across saidcommunication link in response to at least two user inputs, said cameramicrocontroller adjusting a function of said imaging device in responseto a respective one of said at least two voltage levels.
 26. The visionsystem of claim 23, wherein said imaging device comprises a videoencoder, said video encoder communicating said image signal to saiddisplay device via said communication link.
 27. The vision system ofclaim 23, wherein said display device is associated with an interiorrearview mirror assembly of the vehicle.
 28. The vision system of claim23, wherein said display device is associated with an accessory moduleof the vehicle.